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Abstract: The flipped classroom is being increasingly used in a wide range of instructional 
situations, yet little is known about how to facilitate it. The purpose of this study is to explore what 
types of learning activities in a flipped classroom are perceived to be the most effective in the 
achievement of desired course competencies. This case study specifically focused on the classroom 
lab sessions—the student-centered classroom—rather than online self-learning modules. 

Employing a case study using a mixed method approach, this research identifies effective 
pedagogy in facilitating a flipped classroom. Merrill's (2002) first principles of instruction were 
used as a research framework. While results show that students engaged in learning activities of 
demonstration and application, they were barely exposed to higher-order learning activities. That 
leads to the conclusion that implementing problem-centered instructional activities, accompanied 
by desirable challenges, is highly advisable to foster deep engagement. Implications and future 
directions are discussed. 
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Resume : La classe inversee est de plus en plus utilisee dans un large eventail de situations 
pedagogiques, mais on en sait peu sur la fagon de la faciliter. Le but de cette etude est d'explorer 
quels types d'activites dans une salle de classe inversee sont pergus comme etant les plus efficaces 
dans la realisation des competences de cours visees. Cette etude de cas a ete specifiquement centree 
sur des sessions de laboratoire en classe - classe centree sur l'etudiant — plutot que des modules 
d'auto-apprentissage en ligne. En se servant d'une etude de cas tout en utilisant une approche de 
methodes mixtes, cette recherche propose une pedagogie efficace pour faciliter une salle de classe 
inversee. Les premiers principes d'instruction de Merrill (2002) ont ete utilises comme cadre de 
reference pour la recherche. Bien que les resultats montrent que les etudiants participaient a des 
activites d'apprentissage de demonstration et d'application, ils etaient a peine exposes a des 
activites d'apprentissage d'ordre superieur. Cela mene a la conclusion que la mise en oeuvre 
d'activites pedagogiques centrees sur les problemes, accompagnees de defis souhaitables, est 
hautement recommandee pour favoriser l'engagement profond. Les implications et les orientations 
futures sont discutees. 

Mots cles : classe inversee, methodes mixtes, apprentissage centre sur les problemes 


Introduction 

Higher education institutions are facing a great deal of scrutiny for their failure to adequately 
educate students (McLaughlin et al., 2014). Research indicates that higher education 
institutions have not been able to fulfill their role in fostering critical thinking, effective 
interpersonal skills, or reasoning skills—the core competencies intended to be instilled in 
students (Arum, Cho, Kim, & Roksa, 2012). Instead, students are too often disengaged or 
distracted and seemingly lacking in motivation (Abeysekera & Dawson, 2015; Bonk & Khoo, 
2014). 

While new models and pedagogical approaches surrounding the use of online learning 
technology have emerged during the past two decades, little has changed in the structure of 
education (Bonk & Khoo, 2014; Bonk & Zhang, 2008). Face-to-face lectures continue to prevail 
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in a large proportion of classrooms in the United States (Prober & Heath, 2012). Unfortunately, 
findings from research on student attention indicate that the average attention span of a 
student is less than 20 minutes, which results in reduced engagement and interactions, and, in 
effect, highly ineffective use of time for learning (Stuart & Rutherford, 1978). The prevailing 
passive learning model—centered on teacher-directed activities and decisions—deprives 
students of sound educational experiences. 

Among the key concerns in higher education today is that instructors are not tapping into the 
digital learning approaches of their students and they seem in no rush to do so (Schaffhauser, 
2016). Suffice to say, there are many challenges that instructors face in motivating adult 
learners in this new age of digital learning (Kim, 2009; Kim & Frick, 2011). In the midst of such 
ongoing concerns and calls for change within higher education instructional practices, several 
potential solutions have been offered including the implementation of the flipped classroom 
model (Khan, 2012; Strayer, 2007). 

A growing body of literature suggests that flipping the classroom can be a viable alternative to 
facilitating an active learning classroom. In a flipped classroom, students enjoy offloaded 
content at their own pace while class time is dedicated to participating in learner-centered 
activities, such as group projects, discussions, or problem-solving, which are developed 
grounded on an inquiry-based learning approach (Bergmann & Sams, 2012). Instructors play 
the role more of a "guide on the side" rather than "sage on the stage," serving as coach, 
facilitator, or mentor. In this context, their duties center on guiding students in solving 
problems and helping them engage with peers (Reigeluth, 2012). With the flipped classroom 
concept receiving attention in recent years, research is needed to better understand effective, 
efficient, and acceptable pedagogical strategies in this new classroom environment. The 
purpose of this study is twofold. First, we intend to explore types of learning activities in the 
classroom time of a flipped classroom; and second, to identify activities that are perceived by 
students and faculty to be effective in the achievement of desired course competencies. This 
exploratory research is guided by the following two research questions: 

1. What instructional activities were facilitated in the lab sessions? 

2. Were the instructional activities helpful for students to achieve the course 
competencies? 

Literature Review 

Theoretical Framework 

A fundamental premise of the flipped classroom is that migration of lecture materials and 
learner-to-content interaction to a digital online delivery format creates opportunities for 
learning through socially constructed face-to-face instructional events (Tucker, 2012). 
Theoretical justification for the flipped classroom as an effective approach to instruction 
requires a review of student-centered theories. As noted by Bishop and Verleger (2013), 
theories supporting flipped classroom practices revolve around the missed opportunities of 
more traditional instructor-centered classroom models (Pluta, Richards, & Mutnick, 2013). 
Traditional classroom instruction provides higher barriers to the realization of student- 
centered classrooms (Berge, 1998). 

Application of social constructivism principles to the design of flipped instruction affords 
opportunities for the acquisition of knowledge through social interaction, peer feedback, and 
the co-construction of knowledge (Lave & Wenger, 1991; Scardamalia & Bereiter, 2016, 
February 22; Tobias & Duffy, 2009). The constructivist framework suggests a set of 
instructional principles for the design of an effective student-centered learning environment. 
First, all learning activities should be anchored to a larger, meaningful task or problem. 
Learning in context provides opportunities for engagement with content beyond 
memorization. Second, learning activities should support learners' sense of ownership for the 
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overall task. Ownership for learning outcomes refers to strategies that closely align learners' 
personal goals with that of the course. Third, tasks and problems should reflect as much as 
possible an authentic context. Authentic learning activities provide consistent alignment 
between cognitive demands on the learners and on the environment. Fourth, learners should 
develop a sense of ownership of the process used to find a solution to problems. Learners 
should be challenged to think and present their solutions. The instructor's role should 
challenge and support learners' thinking and not dictate and facilitate learners' problem¬ 
solving processes (Savory & Duffy, 1995). While social constructivism defines the 
epistemological foundation for student-centered learning, active learning and problem-based 
learning theories inform in greater detail methods for the design of flipped classroom 
instruction. 

Active learning is broadly defined as any instructional method that increases learners' 
engagement in the learning process, critical thinking and reflection—provided that students 
demonstrate reflective engagement with the content (Prince, 2004). Active learning can occur 
when a teacher stops lecturing and students ask a question or work on a task that is designed 
to help their understanding such as think-pair-share discussion or pair up works, and share 
their answers with the entire class (Andrews, Leonard, Colgrove, & Kalinowski, 2011). Such an 
approach to the design of instruction situates learning within the construction of knowledge 
afforded through peer-interaction and immediate feedback. 

Collaborative learning also plays an important role in supporting flipped classroom methods. 
From an instructional design perspective, collaborative learning requires specific strategies 
supporting positive interdependence, face-to-face interaction, individual accountability, 
interpersonal skills, and group self-evaluation (Doolittle, 1995). Problem-based learning is an 
instructional strategy in which students are asked to present meaningful solutions to 
contextualized, ill-structured problems. Hmelo-Silver (2004) argues that problem-based 
instruction supports student problem-solving skills while re-enforcing the effectiveness of 
collaborative learning. With the above constructs in mind, we utilize Merrill's (2002) first 
principles of instruction, as it provides a framework to evaluate the pedagogical design of 
content aimed at cognitive learning. Merrill's instructional design framework proposes five 
components related to the design of instruction: (1) activation of prior experience, (2) 
demonstration of skills, (3) application of skills, (4) integration of these skills into real-world 
activities, and (5) a problem-centered approach. To be specific, activation helps students learn 
by being directed to recall prior knowledge or experiences. Demonstration can be facilitated 
through presenting new knowledge in the context of real-world tasks and examples. 
Application is promoted when the learner applies new knowledge based on instructor 
feedback. Integration occurs in learners' life through reflection, discussion, debate, and / or 
presentation of new knowledge. And Merrill stated that instructions should be problem- 
centered, implying that students learn more when they engage in relevant real-world tasks or 
problems (Merrill, 2002) 

Research on the Flipped Classroom 

Systematic empirical research on the effects of inverting in and out-of-classroom activities on 
learning outcomes and students' satisfaction is scant (Goodwin & Miller, 2013). According to 
Abeysekera and Dawson (2015), the flipped classroom can be characterized by a change in use 
of out-of-class time and in-class activities that emphasize active learning, problem solving, and 
peer learning. In the literature review conducted by Bishop and Verleger (2013), scholars 
consider the flipped classroom as any course that provides a combination of out-of-class 
instructional tasks, in-class student-to-student activities, and lectures. The themes of empirical 
research on the flipped classroom focused on students' perception of their learning 
experiences. 
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Overall measurements of students' perceptions across studies are relatively consistent (Bishop 
& Verleger, 2013; Day & Foley, 2006; Foertsch, Moses, Strikwerda, & Litzkow, 2002). While the 
majority of participants seem to be better prepared for classroom work after completing the 
homework assignment, a small number of students seem to dislike the flipped classroom 
structure. Face-to-face lectures are preferred over video delivered instruction. Highly 
interactive face-to-face class time is more satisfying than instructor-to-student lectures. Only 
two studies have examined student performance as a function of flipped classroom 
instructional intervention. Both studies show an increase in performance favorable to flipped 
classroom groups (Day & Foley, 2006; Foertsch et al., 2002). 

A more recent review of the scope of flipped classroom research shows inconclusive findings 
on the effectiveness of this approach over conventional teaching methods (O'Flaherty & 
Phillips, 2015). Most flipped classrooms focus on the asynchronous delivery of pre-lecture 
content and in-class, face-to-face synchronous activities (Mason, Shuman, & Cook, 2013; Prober 
& Khan, 2013). Flipped classroom pedagogy must be clearly articulated to faculty and students 
in order to minimize confusion with the learning process (Ferreri & O'Connor, 2013; Mason et 
al., 2013). While some studies report increased student satisfaction with the flipped classroom 
approach, few articles used rigorous methodology to evaluate educational outcomes 
(O'Flaherty & Phillips, 2015). 

Results from the review of the flipped classroom literature suggest that there is a lack of 
consensus about the best combination of instructional principles that most effectively integrate 
in- and out-of-classroom activities. There also seems to exist across studies a lack of clarity on 
specific tasks and activities that can best elicit the advantages of the constructivist approach to 
learning. It is the purpose of this paper, therefore, to begin to address these gaps in the 
research literature by exploring effective ways in which to structure learning tasks in the 
context of flipped classroom instruction. 

Method 

Mixed Methods Approach 

This study employed a mixed methods approach, a research design that uses both quantitative 
and qualitative data to answer a particular question or set of questions (Hanson, Creswell, 
Clark, Petska, & Creswell, 2005). The utilization of a mixed methods approach can provide 
several benefits, including triangulation, complementarity, development, initiation of another 
question, and expansion (Greene, Caracelli, & Graham, 1989). Qualitative data were gathered 
through a series of observations of classroom activities and a semi-structured open-ended 
interview with the instructor, while results from a Likert-scale survey generated data for 
quantitative analysis. The processes of data collection and analyses were grounded on Merrill's 
(2002) First Principles of Instruction—(1) Task/Problem-Centered, (2) Activation, (3) 
Demonstration, (4) Application, and (5) Integration. 

Context and Participants 

Flipped classrooms are composed of two primary sections—(1) Self-paced online learning out- 
of-class (i.e., lecture videos), and (2) face-to-face sections for engaging in instructional 
activities. The course was a graduate public health class, addressing mostly foundational 
theories and models used in the field of public health. The class was a 3-credit required course 
for first-year master's students, and contained 14 online modules and two exams. Starting in 
Fall 2014, the primary instructor decided to flip the class by providing weekly learning 
modules in Canvas, which was adopted as the school-wide learning management system. Each 
weekly module provided a brief task and learning-related instructions, an hour-long self- 
paced online lecture, and supplementary resources such as readings. The preloaded self-paced 
lectures were designed and developed by two experienced instructional designers and the 
course instructor using an e-learning authoring tool. Articulate Storyline. Each lecture included 
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five main sub sections; (1) warm-up, (2) learning objectives, (3) lessons and self-assessment, (4) 
activities, and (5) summary. Such activities primarily took place in the classroom before 
flipping the class. It was the students' responsibility to watch the weekly online lectures and 
finish assigned readings before coming to the labs so that they could engage in deep 
discussions and apply the lecture content covered in the online lessons. 

The participants in the study were 70 graduate students enrolled in the course and the 
instructor for the introductory graduate course. The instructor had sufficient expertise in the 
public health knowledge base and classes using face-to-face pedagogical instruction, yet she 
was new to the flipped classroom format. 

Data Collection and Analysis 

We used three data sources—(1) classroom observation, (2) a student survey, and (3) an 
instructor interview. All three data were used to address the first research question (What 
kinds of instructional activities were facilitated in the lab sessions?) while only survey data 
were used for the second research question (Were the instructional activities helpful for 
students to achieve the course competencies?). 

Classroom Observation. Four researchers conducted three non-intrusive observations of lab 
sessions to identify the kinds of instructional activities that were being facilitated in the lab 
session. Each laboratory session was designed to lead students through several in-class 
activities and lasted approximately 60 minutes. During observation, researchers produced field 
notes along with timestamps describing the instructor and student's actions for each event. 
Upon completion, classroom events were coded according to Merrill's (2002) five principles of 
instruction. According to Armstrong, Gosling, Weinman, and Marteau (1997), inter-rater 
reliability measurements strengthen the validity and reliability of qualitative studies by 
ensuring agreement across multiple coders. Since four researchers were involved in 
observations, it was important to ensure inter-rater reliability by computing Fleiss' Kappa 
value (Fleiss & Cohen, 1973). We conducted three observations, which required three 
computational processes. As a result, we obtained .693, indicating substantial agreement with 
observers' interpretations on instructional activities based on the five principles of instruction. 
Table 1 provides details. 


Table 1. Inter-rater Reliability 



Lab 1 

Lab 2 

Lab 3 

Average 

Percentage of agreement 

72.0 

92.3 

69.02 

77.77 

Kappa value 

.611 

.893 

0.574 

0.693 


Student Survey. We administered the survey to 70 graduate students, asking about their 
perception of how well learning activities were facilitated in labs, and how much those 
learning activities helped in the achievement of course competencies. Our survey 
questionnaire was based on the course evaluation instrument for assessing Teaching and 
Learning Quality (TALQ) (Frick, Chadha, Watson, Wang, & Green, 2009). TALQ synthesized 
different learning frameworks for addressing student learning and teaching from multiple 
points of view but we focused only on questions related to Merrill's first principles of 
instruction in a course (Merrill, 2002; Merrill, Barclay, & Van Schaak, 2008). In total, 15 5- 
Likert-scale questions were developed, with the last question being open-ended. We included 
sub-questions for each item, asking whether or not an activity was perceived as helpful in 
achieving course competencies. For validity of the questionnaire, we also compared the TALQ 
questions with the original framework proposed by Merrill (2002). 
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Instructor Interview. At the end of the semester, the four researchers conducted a semi- 
structured interview for an hour with the instructor of the course. To gain in-depth data about 
how the instructor felt about the flipped classroom format, we asked questions about the 
instructor's perceptions of achievement of learning outcomes, effectiveness of the activities, 
barriers, and benefits based on the Merrill's (2002) first principles of instruction. In responding 
to both in-general questions and incident-oriented questions, the instructor was able to recall 
sufficient diverse events and perceptions of the class. Upon completion of the interview, the 
four researchers conducted thematic analysis and several important themes were highlighted 
from the thematic analysis (Braun & Clarke, 2006). 

Results and Findings 

We provide the results and findings that followed from the data collection. The first question is 
addressed by classroom observations, student survey, and instructor interview data while the 
second question is explained only by the survey results. Again, Merrill's first principle of 
instruction is used to organize our findings. The five components of Merrill's first principle 
are: (1) activation of prior experience, (2) demonstration of skills, (3) application of skills, (4) 
integration of these skills into real-world activities, and (5) problem-centered. 

What Instructional Activities Were Facilitated in the Lab Sessions? 

Observational data illustrate that the most frequently used principle in the lab was 
'application.' The instructor spent 35% of lab time applying the concepts, rules, and 
procedures of the lecture. Most of the application activities were 5-10 minutes long and were 
conducted as small group or pair-up discussions about what the students had learned in the 
lecture so that they could practice or try out what they had learned in the lab. After each 
activity, students shared their discussions with the class, and the instructor gave feedback and 
wrapped up the discussion. All four researchers agreed that the instructor had been successful 
in enhancing student engagement in discussion. Many of the pair-up and group discussion 
activities required students to do something, such as complete a worksheet to earn 
participation points. All lab sessions we observed had an active atmosphere. Students though, 
seemed to rush in completing the tasks, since the lab sessions lasted only 60 minutes. Overall, 
the pace of the labs was fast yet engaging. For the fast-paced classes, it was essential to stay on 
track to complete the classroom tasks. Even so, the instructor did not neglect those who came 
in late, although she had many activities to facilitate in a limited amount of time. Content was 
domain-specific, but not too knowledge-heavy. Students' responses to open-ended survey 
questions indicated that the concepts were easy-to-follow without many challenges 
encountered. 

'Demonstration' was the second most frequently used principle (30% of lab time) from the 
observation data. The instructor provided students with the main concepts of the weekly 
lessons to help students refresh their memory of online lectures and retain information. The 
instructor utilized a wide range of hands-on examples and multimedia resources to help 
students understand the concept. She then let students think of related examples for 
comparing and contrasting the concepts. She also provided hands-on examples to address new 
concepts and theories. According to responses on the open-ended student survey questions, 
students found hands-on examples helpful and relevant to their lives. 

'Integration' activities were also conducted to provide students an opportunity to explore how 
they could apply what they had learned to the real world (21%). Although relatively less 
occupied in the class due largely to the abstract nature of the content—public health theories 
and models—integration activities were provided whenever relevance emerged. 

Most 'activations' were observed at the beginning of the lab (14%). Activation-related 
instructional activities were mostly done in group or paired-up discussions. One sample 
activation activity addressed the importance of a social network for health. Students were 
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given a piece of paper to write down the names of acquaintances and friends with whom they 
were connected to visualize their social network. The instructor provided gentle sentimental 
background music while they engaged in this activity. Due to the emotional relevance, 
students appeared immersed and deeply engaged in the activity. 

No activities related to 'problem-centered' were observed in the three class sessions. The 
interview and survey data indicated a small portion of 'problem-centered' activities in non¬ 
observed class sessions. Figure 1 summarizes the instructional activities observed during the 
three class sessions. 



* Problcm-ccntcrcd: 

No activities were observed 


Figure 1: The average portion of each category from the three labs. 

While the results of the survey illustrate the effectiveness of learning activities and facilitation 
from the students' perspective, the interview with the instructor added additional points about 
the five principles from the instructor's point of view. 

The result of the three survey questions associated with 'demonstration' indicated that the 
instructor showed a proficient facilitation of 'demonstration' in the lab (4.15 out of 5). Students 
reported that the instructor demonstrated skills in fostering learning objectives in the labs by 
providing examples and counter-examples. Also, the instructor provided alternative ways of 
understanding the same ideas or skills. Given that demonstration is the second most 
frequently used principle according to the observation (30% of lab time), we came to 
understand that the instructor placed an emphasis on demonstration activities from the 
instructor interview. For example, the instructor stated, "I did use some media to present new 
knowledge. I had some video clips, and when possible I actually tried to have students, be the 
ones to generate that knowledge." 

In terms of 'activation', survey data revealed that 'activation' was well facilitated in the lab 
(4.11 out of 5). Students strongly agreed the instructor provided a learning structure that 
helped them to mentally organize new knowledge and skills and also allowed students to 
connect past experience to new ideas and skills. The instructor also made efforts to activate 
student prior knowledge. According to the instructor, 

I did try to do that in bring in something that would already be familiar and then build 
on that with some of the newer content. For example, before I explained the conceptual 
works of public health, I let the students draw their own social networks so that they 
would realize how many people are actually in their networks. 

Seeing the scores of 'application' (3.93 out of 5), we can interpret that students had 
opportunities to practice or try out what they learned in this course but the students did not 
feel 'application' activities were facilitated well, which is directly contradictory to the 
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instructor's emphasis on the labs. In particular, the score of the questions, "My course 
instructor gave me personal feedback or appropriate coaching on what I was trying to learn," 
and "the instructor detected and corrected errors I was making when solving problems, doing 
learning tasks, or completing assignments" were relatively low compared to other items (3.82 
out of 5). The somewhat low score was an unexpected finding, as we all observed that the 
instructor endeavoured to provide as many hands-on activities as possible. Here is the 
instructor's reflection on 'application' activities: 

Application was at least to me the most important part of the labs and something that I 
tried to do every single time. I think every lab they would break up into groups.. .So, 
applying the content is something very tangible and that was actually something we did 
like I think in every lab. 

The discrepancy between the instructor and students' perception on 'application' activities 
might be explained by the formatting of the open-ended questions on the survey. For example, 
many students reported that the lab sessions were too short and they did not have sufficient 
time to digest the content. Quoting one student, "Application was the most important part of 
the labs, but just doing simple activities with no challenges is not very helpful for learning". 
This may be related to the high degree of motivation and the appropriate amount of time 
required for processing new concepts and knowledge. Also, students and the instructor both 
found insufficient time for "Just-in-Time" learning and interaction. Moreover, students 
reported that the activities and exercises were not challenging, resulting in the decrease of 
student engagement and motivation. 

Also, the instructor reported challenges with student accountability. The lab sessions were 
designed to provide authentic tasks in alignment with the online lectures. The instructor found 
that some students did not watch the lecture online and thus had difficulty in engaging in the 
lab sessions, which later resulted in ineffective learning. Quoting the instructor, 

...Holding students accountable for things is very important. I know there is a little bit of 
a divide between they're adults and they're in charge of their learning. So, they get to 
decide, whether they're going to participate, or watch the content, or do the reading, or 
whatever it might be. 

For the 'integration' (3.97 out of 5), they agreed that the lab activities allowed them to reflect 
on, discuss with others, and defend what they learned. But students indicated that there was 
not enough opportunity to publicly demonstrate to others what they had learned in the course. 
In contrast to 'application', the instructor believed that she did not have much chance to 
promote 'integration,' which, she assumed, might be addressed when students would 
complete internships and the culminating experience. In the interview, the instructor indicated 
that she seldom asked questions associated with integration. For 

The instructional principle that received the lowest number was 'problem-centered' learning 
(3.43 out of 5). As noted above, 'problem-centered' learning was not observed in the three class 
sessions, and the low survey score would suggest it was the activity least often used in the rest 
of the lab sessions. 

Centered on her teaching philosophy, the labs were facilitated with the intention of making the 
best use of lab time to apply what students learned in the online self-paced online modules 
and to have a deep discussion with others. Themes from the interview with the lab instructor 
were: active learning, engagement, positive learning, development of skill sets, awareness of a 
need to align with learning objectives, retention of knowledge, and consistent structure of lab 
sessions. The instructor's active learning and engagement approach was well reflected during 
the conversation. Even while demonstrating, the students were encouraged to think of hands- 
on and related examples, and they were asked to do something after presenting new concepts 
or theories. Pair-up or group works were the main methods for synthesizing concepts while 
they articulated the thinking process and results of the cases (Andrews et al., 2011). 
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Figure 2 shows student perception of lab activities based on first principles of instruction. 


5.00 

4.00 3.43 

3.00 

2.00 j 
1.00 
0.00 • 

Problem-centered 


4.11 4.15 


Activation Demonstration 


3.93 3.97 


Application Integration 


Figure 2: Student perception of lab activities. 

Centered on her teaching philosophy, the labs were facilitated with the intention of making the 
best use of lab time to apply what students learned in the online self-paced online modules 
and to have a deep discussion with others. Themes from the interview with the lab instructor 
were: active learning, engagement, positive learning, development of skill sets, awareness of a 
need to align with learning objectives, retention of knowledge, and consistent structure of lab 
sessions. The instructor's active learning and engagement approach was well reflected during 
the conversation. Even while demonstrating, the students were encouraged to think of hands- 
on and related examples, and they were asked to do something after presenting new concepts 
or theories. Pair-up or group works were the main methods for synthesizing concepts while 
they articulated the thinking process and results of the cases (Andrews et al., 2011). 

In summary, data from observations suggest that 'demonstration (30%)' and 'application 
(35%)' were the two most frequently facilitated activities but the survey result shows that 
students perceived 'demonstration (4.15)' and 'activation (4.11)' were well facilitated, and that 
eventually helped them achieve course competencies. Problem-centered instruction was the 
least often used and not effectively facilitated (3.43). 

Were the Instructional Activities Helpful for Students to Meet the Learning Objectives? 

We have reported, based on observations, survey, and interview, which instructional activities 
were most often facilitated in the lab sessions. In this section, we report how these activities 
helped students meet the course competencies based on the survey data. Descriptive statistics 
of the survey indicates that problem-centered activities were not very well facilitated (3.43), 
but those were helpful in achievement of course competencies (3.63). 'Activation (4.14)' and 
'demonstration (4.05)' were shown to be very well facilitated and helpful in the achievement of 
desired outcomes. 'Application' was rated somewhat lower (3.97), which is attributable to the 
short length of lab sessions, easy tasks, and one-time (consumable) activities rather than a 
long-term project (e.g., capstone project), evidenced by survey open-ended questions. 

Figure 3 shows the survey results on perceived effectiveness on facilitation and helpfulness of 
learning activities in labs. 
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Figure 3: Survey result on perceived effectiveness on facilitation and helpfulness. 

Correlation and multiple regression analyses were also performed to examine the relationship 
between the five principles of instruction and the degree of perceived helpfulness in the 
achievement of course competencies. The result of the multiple regression model with all five 
predictors results in students' experiencing the learning activities based on the five principles 
of instruction are positively correlated with the achievement of course competencies (R 2 = .804, 
p < .001). Table 2 summarizes the multiple regression analysis results. 

Looking at the results in detail, 'Application' is the only independent variable that is not 
statistically significant, although 'application' activities were the center of the labs. Students 
reported that 'application' activities were relatively less helpful in achieving desired course 
competencies. Thirteen students' responses in open-ended questions in the survey partially 
explain that application activities were too short to deeply think about the problem with no 
challenges. This suggests that the class should have implemented more real-world practice and 
in-depth projects so that students could appreciate appropriate challenges that foster 
knowledge development. One student wrote "...I felt that making the activities more about 
real-world application or more of an interactive case study would be more helpful". 

Table 2. Multiple-regression Analysis 


Model Summary 


Model 

R 

R Square 

Adjusted R Square 

Std. Error of the 

Estimate 

Durbin-Watson 

1 

.897 a 

.804 

.785 

1.17208 

2.178 


Coefficients 



Unstandardized Coefficients 

Standardized 

Coefficients 



Model 

B 

Std. Error 

Beta 

t 

Sig. 

1 (Constant) 

2.260 

1.324 


1.707 

.094 

Problem-centered 

.738 

.231 

.241 

3.190 

.002 

Activation 

.987 

.391 

.241 

2.528 

.015 

Demonstration 

1.395 

.447 

.289 

3.122 

.003 

Application 

.511 

.325 

.133 

1.573 

.122 

Integration 

.810 

.260 

.239 

3.113 

.003 
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Discussion 

In our case, the lab was intended to provide an authentic learning experience with hands-on 
examples for the online component; application was an important component to be addressed. 

Key takeaways were drawn from data analysis. Firstly, data suggested that 'demonstration' 
occurred more than the 'problem-solving' approach. From a conversation with the instructor, 
we believe providing required instructional activities that check learners' understandings on a 
weekly basis would be beneficial. However, thirty percent of lab time used for demonstration 
activities is significant, leaving insufficient time for deep interactions and discussions. 

Secondly, while the labs offered a variety of instructional activities that help students engage 
with each other, observational, interview, and survey data all indicated that students were 
focused on completing tasks. One of the interesting findings from the open-ended questions of 
the survey stated that students felt they were mostly engaged in understanding-check 
activities rather than knowledge-developing activities during the lab sessions. Students 
confirmed this by saying they did not feel a significant challenge in their learning process, 
which we interpret as the main cause of the lack of desirable difficulty that later directly 
influenced student motivation (McDaniel & Butler, 2011; McDaniel & Einstein, 2005; Metcalfe, 
2011; Yue, Bjork, & Bjork, 2013). The flipped classroom is supposed to provide students with 
opportunities for a knowledge-constructing process (Michael, 2006). 

One area for improvement though might take into account that the class was not very 
problem-centered. Much class time was used for checking the understanding of student 
knowledge, but with few challenges. This may have led to the inability to fulfill the 
fundamental purpose of the class, requiring problem-centered instructional strategies 
embedded in the semester. 

In addition to the lack of problem-centeredness, one of the major concerns raised by both 
students and the instructor was the time constraints built into the course. Students reported 
that attending a 60-minute-long face-to-face lab session once a week was not sufficient to 
achieve in-depth knowledge of theories and models introduced in the online lectures. This led 
to reduced time for providing personalized feedback for individual students and relatively less 
attention to monitoring the individual learning process. This student-instructor ratio and the 
time constraints call for a strong need to contemplate an alternative avenue to fulfill the need 
for an application-oriented class with an individualized learning plan using current resources. 

One way to address the time constraint problem is to use both online spaces and the face-to- 
face classroom as an active place for learner interaction. In this class, students were required to 
watch the preloaded lecture module and read assigned articles outside of class. In addition, 
they were encouraged to be more active in online space. Teachers also use online space as a 
tool to check student understanding so that they provide tailored just-in-time instruction based 
on student questions through online prior to class (Berrett, 2012). Facilitating a weekly 
assessment of online lectures in the formats of reflective writing, discussion forums, or creating 
artifacts will maintain a sound degree of tension that can help learners routinize the format of 
the course. 

Another effective way to address the lack of problem centeredness could be the 
implementation of the Problem-Based Learning (PBL) approach (Hmelo-Silver, 2004; Hmelo- 
Silver & Barrows, 2006, 2008; Hmelo-Silver, Duncan, & Chinn, 2007). Unlike the traditionally 
formatted classroom, the PBL approach enables instructors to facilitate a learner-centered 
classroom where the learners are highly responsible for their own learning. We suggest 
including instructional activities that promote deep learning and thoughtful reflection on 
content. 

A positive learning culture is considered a catalyst for the flipped classroom. This comes from 
a highly interactive class where learners engage in problems, projects, and peers. Hence, the 
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effectiveness of such a format for instruction is expected to enhance students' utilization of the 
classroom as a place of discussion. Here the learning culture comes into play through 
encouragement of a fear-free environment while a supportive peer-learning classroom 
becomes a prime need. 

Recognizing the lack of 'problem-centered' activity, we believe it is advisable to embed a 
project-based learning process with milestones to achieve, requiring a group effort to create 
something so that students strive to maximize their learning experiences. The frequent use of 
application activities fulfilled the main purpose of the labs, as corroborated by the instructor 
and students. However, the provision of personal feedback in place of group feedback is 
needed so that students can better internalize and digest learning content at their own pace. 
Interestingly, the students wanted a period longer than 60 minutes for their lab sessions. 
Students needed longer time for reflection on the content, an active attitude that impressed all 
of us. 

Implication 

The flipped classroom shows promise for student-paced learning, utilizing class time for deep 
learning activities, flexible customizability of curriculum, and helping learners become self- 
directed learners with the support of technology. The flipped classroom is intended to enhance 
students' higher-order thinking, critical thinking, and collaboration competencies. On the other 
hand, it requires greater preparation time, and involves student resistance to a new approach, 
accountability, and content coverage. Flipping the class is about redesigning courses, which 
requires a considerable amount of time and resources. Higher education institutions are in 
need of responding to different educational needs by providing appropriate support. 
Furthermore, pedagogical support in alignment with technology is necessary. Providing 
sufficient student support for effective learning is another important aspect, and instructional 
design support in the course preparation phase is essential. 

To address this, strong motivation of teachers is a must in the preparation of a well-designed 
course, and this is enhanced by institutions' efforts and support. Recognizing that the flipped 
classroom has not yet established a strong research base, more empirical research needs to be 
conducted to elaborate the approach. Collaboration between institutions, faculty, and students 
is an urgent need. 

Conclusion 

This study intended to begin to identify a set of sound instructional design principles for a 
face-to-face section of flipped classroom context. In this study, lab sessions utilized mostly 
'application' and 'demonstration', which is aligned with the purpose of labs in the flipped 
classroom setting. However, most of the instructional activities in the lab were expendable 
rather than continuous, leaving a short period of time for students to engage in activities. 
Naturally, students seemed not deeply engaged in activities, resulting in perceptions of 
learning without challenges. This is largely attributable to the large proportion of 
'demonstration', in other words, lectures and understanding check-ups, which are rather low- 
order thinking activities (Bloom, 1956). Also, one of the key lessons learned from the case 
include the importance of 'accountability' and a consideration of required assessments that 
monitor student's learning progress. An effective use of lab time might incorporate a semester- 
long PBL approach so that students collaboratively learn from each other by solving real-world 
problems and creating artifacts in association with the concepts. In doing so, it is important to 
maintain 'desired difficulty' to maintain the high motivation of the learners. One limitation of 
this study is that we only investigated the lab sessions. Since the online lectures and lab 
sessions are intertwined, it is also important to address what happened in the lecture sessions. 
Therefore, future research should address the link between the two sections. 
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This case study is exploratory in nature and much remains undiscovered in the literature on 
the flipped classroom. Although more exploratory studies need to be done for theory 
development in the flipped classroom, future research should also employ controlled studies 
that investigate student performance throughout a semester using a range of measurement 
tools. Faculty, instructional designers, curriculum designers, and trainers would benefit from 
the flipped classroom research, which clearly articulate the relations between effects and 
activities. 
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